Apple VRML Binary
Proposal
Draft 0.6 - - - April 29, 1996
Fábio
Pettinati		 Ronie
Hecker		 Pablo
Fernicola		 Klaus
Strelau


1. Introduction

The Apple VRML Binary metafile embodies the semantics of VRML 2.0 and relies on the architecture of Apple Computer's 3DMF (3D Metafile Format) binary metafile format to generate a file format that addresses the issues in the "Requirements for the Binary Encoding of VRML 2.0" document published by the VRML Architecture Group in March 1996.

2. Structure of a VRML Binary Metafile

Every VRML binary metafile contains the following components:

Header Object 0 Object 1 ... Object n Null Object

The Null Object is a special object that delimits the end of a metafile (EOF). All object types are described in detail later in this document.

2.1 VRML Binary Header

The binary VRML metafile contains the following header:

# V R M L   [ d r a f t N ]   v2 . 0 b i n a r y

where N represents the draft number. The [draftN] syntax is used for the draft version of binary VRML. The header for the final specification of binary VRML is:

# V R M L   v 2 . 0  b i n a r y

In either case, the end of the header is marked by the newline character '/n'. The newline character denotes the end of the header information. The byte following the header is the most significant byte of the first object in the metafile.

2.2 VRML Binary Object Structure

Each object in a metafile starts with a byte containing a Control Code denoting the object type. The Control Code is followed by one to four bytes containing the total number of bytes that follow, and that comprise the object's content. The Control Code encodes the object type in its upper four bits. The lower two bits encode the number of bytes (1, 2, 4, or 8) necessary to represent the size of the object's content. The byte following the field containing the object's size is the first byte of the object's content. Every object in a VRML binary metafile, with the exception of the Null Object, is required to have its size present after the control code byte.

Control Code Size Info Object Content
Type Code unused Size
Code
XXXX X X 0 0 1 byte "size"bytes with object content
XXXX X X 0 1 2 bytes "size"bytes with object content
XXXX X X 1 0 4 bytes "size"bytes with object content
XXXX X X 1 1 8 bytes "size"bytes with object content

Size information in a VRML binary metafile is important for several reasons. Parsers can use an object's size information to pre-allocate contiguous memory space to hold that obejct's data; a priori memory allocation can lead to significant performance improvements. Parsers can also use size information to quickly scan a binary metafile. This can prove very useful, especially when invalid information is present in a binary metafile.

Parsers can deal with incorrectly generated metafiles by comparing Control Codes against a table of registered values. A metafile that has unknown Control Codes is deemed invalid. This spec prescribes no error recovery on the parser's part. However, well-behaved parsers are expected to use the obejct size field to skip over unknown object types.

Although skipping is not a reliable error recovery mechanism, the idea is that, for minor errors, an invalid VRML binary metafile can still be rendered. Abruptly quitting on the user when a parser encounters an invalid VRML binary metafile is highly frowned upon!

2.3 Defining Fields Inside Objects

The VRML 2.0 specification states that, in a metafile, fields inside an object are optional, i.e. their presence is only required to override default values specified in that object's definition. Furthermore, the ASCII file format specification for the ASCII file format states that the order in which fields are specified is not important. The VRML binary metafile specification preserves the notion of optional fields, but requires that fields in a file be specified according to their canonical order based on an object's definition. The appendix in this document contains a list of all fields and their canonical order for all built-in nodes.

Inside a VRML binary metafile, we use a variable-length bit array to indicate which fields are present. This array is packed as a sequence of bytes (i.e. the bit array is padded to an 8-bit boundary to keep the file character-aligned.) This array is preceded by a 1-byte field containing the total number of bytes in the array itself, from 1 up to 255. This scheme allows for arrays definining up to 2040 (8 * 255) fields.

A 1-byte long bit array indicates the presence of up to 8 fields. A 2-byte long bit array indicates the presence of up to 16 fields. A 4-byte long bit array indicates the presence of up to 32 fields. A 8-byte long bit array indicates the presence of up to 64 fields.

This bit array is encountered from the most significant byte (highest index) to the lowest significant byte (lowest index) in the metafile. Each bit that is '1' indicates that the field corresponding to that bit's index is present in the metafile. A one-byte long bit array of all zeros indicates that no fields are present inside a given object.

Following the bit array, are the fields specified in the metafile. The first field has an index equal to the index of the lowest set bit in the bit array. The second field has an index equal to the index of second lowest set bit in the bit array. In other words, the fields are encountered in their canonical order in the object definition.

Here are examples of 1-, 2-, 3-, and 4-byte long bit arrays:

Byte 0
f 7 f 6 f 5 f 4 f 3 f 2 f 1 f 0

Byte 1 Byte 0
f 15 f 14 f 13 f 12 f 11 f 10 f 9 f 8 f 7 f 6 f 5 f 4 f 3 f 2 f 1 f 0

Byte 2 Byte 1 Byte 0
f 23 f 22 f 21 f 20 f 19 f 18 f 17 f 16 f 15 f 14 f 13 f 12 f 11 f 10 f 9 f 8 f 7 f 6 f 5 f 4 f 3 f 2 f 1 f 0

Byte 3 Byte 2 Byte 1 Byte 0
f 31 f 30 f 29 f 28 f 27 f 26 f 25 f 24 ...... f 7 f 6 f 5 f 4 f 3 f 2 f 1 f 0

2.4 Specifying Variable-Length Data

The very nature of a binary file format dictates that information be encoded so as to occupy as few bytes as possible. Indices or reference values are a good example. One could use a fixed set of bytes (usually four) to represent all index values. This scheme works but often wastes spaces. A better scheme is to use as few bytes as possible to represent a given value, that is, resorting to variable-length fields.

In a VRML binary metafile, variable-length fields are contained in 1, 2, 4, or 8 consecutive bytes, with the two most significant bits of the first byte encoding the total number: '00' for 1 byte, '01' for 2 bytes, '10' for 4 bytes, '11' for 8 bytes. The following table illustrates how this works:

Byte 0
0 0 X X X X X X

Byte 1 Byte 0
0 1 X X X X X X X X X X X X X X

Byte 3 Byte
2		 Byte
1		 Byte 0
1 0 X X X X X X ... ... X X X X X X X X

Byte 7 Byte
6		 Byte
5		 Byte
4		 Byte
3		 Byte
2		 Byte
1		 Byte 0
1 1 X X X X X X ... ... ... ... ... ... X X X X X X X X


2.5 VRML Binary Metafile Primitives

A VRML Binary metafile contains the following primitives: VRML binary metafiles containing on-the-fly node definitions need to encode field information for later use. The following table summarizes the encoding for all primitive types:

Code Field Type Size (bytes) Values
0x00 SFBool 1 0x00 or 0x01
0x01 SFColor 2 to 12 RGB color information
0x02 SFFloat 4 IEEE single-precision floating point value
0x03 SFImage Variable Image data
0x04 SFInt32 4 value in [-2**32 , 2**31]
0x05 SFNode Variable Node control code and associated data
0x06 SFRotation 4 to 16 rotation axis and angle information
0x07 SFString Variable String contents in UTF-8 format
0x08 SFTime 8 IEEE double-precision floating point value
0x09 SFVec2f 8 2 IEEE single-precision floating point values
0x0A SFVec3f 2 to 12 3 IEEE single-precision floating point values
0x81 MFColor n * 2 to n * 12Multiple SFColor values
0x82 MFFloat n * 4 Multiple SFFloat values
0x84 MFInt32 n * 4 Multiple SFInt32 values
0x85 MFNode Variable Multiple SFNode values
0x86 MFRotation n * 4 to n * 16 Multiple SFRotation
0x87 MFString Variable Multiple SFString values
0x88 MFTime n * 8 Multiple SFTime values
0x89 MFVec2f n * 8 Multiple SFVec2f values
0x8A MFVec3f n * 2 to n * 12 Multiple SFVec3f values

2.6 VRML Binary Metafile Built-In Nodes

The VRML specification defines a number of built-in node types that parsers are supposed to know about. The VRML binary metafile specification represents these built-in nodes in encoded form, rather than using their names; this saves considerable space, and also helps parsing considerably. The following Built-In Node Table shows the encoding for all VRML built-in node types:

Built-in Node Code String Value
Dec Hex
00x00 Anchor
10x01 Appearance
20x02 AudioClip
30x03 Background
40x04 Billboard
50x05 Box
60x06 Collision
70x07 Color
80x08 ColorInterpolator
90x09 Cone
100x0A Coordinate
110x0B CoordinateInterpolator
120x0C Cylinder
130x0D CylinderSensor
140x0E DirectionalLight
150x0F DiskSensor
160x10 ElevationGrid
170x11 Fog
180x12 FontStyle
190x13 Extrusion
200x14 Group
210x15 ImageTexture
220x16 IndexedFaceSet
230x17 IndexedLineSet
240x18 IndexInterpolator
250x19 Inline
260x1A LOD
270x1B Material
280x1C MovieTexture
290x1D NavigationInfo
300x1E Normal
310x1F NormalInterpolator
320x20 OrientationInterpolator
330X21 PixelTexture
340x22 PlaneSensor
350x23 PointLight
360x24 PointSet
370x25 PositionInterpolator
380x26 ProximitySensor
390x27 ScalarInterpolator
400x28 Script
410x29 Shape
420x2A Sound
430x2B Sphere
440x2C SphereSensor
450x2D SpotLight
460x2E Switch
470x2F Text
480x30 TextureTransform
490x31 TextureCoordinate
500x32 TimeSensor
510x33 TouchSensor
520x34 Transform
530x35 Viewpoint
540x36 WorldInfo
55 0x37 RESERVED for future use
56 0x38 RESERVED for future use
57 0x39 RESERVED for future use
58 0x3A RESERVED for future use
59 0x3B RESERVED for future use
60 0x3C RESERVED for future use
61 0x3D RESERVED for future use
62 0x3E RESERVED for future use
63 0x3F RESERVED for future use

2.7 Data Compression

Data compression is an integral part of the VRML Binary Metafile Specification. The current version of VRML binary metafile specification only considers lossy compression on a per field basis, that is, individual fields within an object are compressed whenever it makes sense. Future versions of the specification might include additional forms of compression, including lossless compression at a field, object, or file level. The following VRML fields can be compressed using a variety of schemes: SFColor, MFColor, SFRotation, MFRotation, SFVec2f, MFVec2f, SFVec3f, MFVec3f. A metafile provides compression information to parsers via the Compression Object. A VRML binary metafile may contain multiple compression objects. Each occurrence of a compression object specifies the compression scheme for a given field type. All fields of that type, occurring later on in the metafile will use the current compression scheme. Changing to a new compression scheme for a given field type requires the specification of another compression object. Using compression objects allows us to specify multiple compression schemes for the same field type. For example, a three-dimensional normalized vector (SFVec3f) can be represented with no compression (using 12 bytes), low compression (using 3 bytes), and high compression (using 2 bytes). With low compression, the median angular error is 0.05 degrees, and with high compression the error is 0.45 degrees.

2.8 Parsing and System Endianness

The binary metafile is a collection of objects in byte-code sequence. The file is parsed as bytes, which denote possibly larger structures. No alignment of larger primitive data types (e.g. floating point, 32-bit integers) is enforced. All primitive data types are always specified from the most significant byte to the least significant byte. Parsers deal with alignment issues when converting to and from multi-byte data types as a function of processor endianness. As part of their initialization sequence, parsers must determine whether they are running on a big-endian or little-endian system. Parsers on different processors deal with endianness issues by loading values in the correct order. This can be accomplished with no performance hit on either system. Here is an example of how a parser would read a four-byte long signed integer on a big-endian system: 
char *ParseLong(char *inBytes, long *theValue)
{
    *theValue = (inBytes[0] << 24) |
                (inBytes[1] << 16) |
                (inBytes[2] <<  8) |
                (inBytes[3] <<  0);
    
    return (inBytes + 4);
}
Here is an example of how a parser would read a four-byte long signed integer on a little-endian system: 
char *ParseLong(char *inBytes, long *theValue)
{
    *theValue = (inBytes[0] <<  0) |
                (inBytes[1] <<  8) |
                (inBytes[2] << 16) |
                (inBytes[3] << 24);
    
    return (inBytes + 4);
}

3. VRML Binary Object Definition

Each object in a VRML Binary metafile starts with a one-byte long Control Code. Each Control Code defines the type of the object being parsed; it also contains information that is required to parse that object. The remainder of this section contains the definitions for all object types in a VRML binary metafile.

3.1 Null Object

The Null Object marks the end of a VRML binary metafile. Remember "Highlander", that awesome movie starring Sean Connery? Then repeat after me: "There Can Be Only One!" Anything else following a Null Object will be ignored. The Null Object contains only its Control Code.

Control Code
0x0 0x0

3.2 Compression Object

The Compression Object specifies how certain objects fields can be compressed using different compression schemes or strategies. The idea is that each occurrence of a compression object resets the compression state of up to 255 field types. At least one field has to be specified by each occurrence of a compression object. Each compression object contains one byte with the total number of fields being specified, followed by that number of pairs of bytes, each one containing field compression information. The first byte in a pair contains the field index, and the second byte contains the compression scheme code. This gives room for up to 256 field types, and 256 compression schemes. Here is the basic layout for a compression object:

Control Code Size Info Num Fields Field Index Compr. ... ... ... Field Index Compr.
0x1 ... ... ... ... ... ... ... ... ... ...

Here is the compression table containing field indices and compression schemes:

Index Field Description Compression Scheme Compression Description
0 Normalized value in [0, 1] 0 Uncompressed (4 bytes)
1 [0, 65535] (2 bytes)
2 [0, 255] (1 byte)
1 SFColor 0 Uncompressed (12 bytes)
1 R-G-B -> 8-8-8 (3 bytes)
2 R-G-B -> 5-5-5 (2 bytes)
2 Normalized SFVec3f 0 Uncompressed (12 bytes)
1 0aaabbccddeeffgg...kk (3 bytes)
2 0aaabbccddeeffgg (2 bytes)
3 Rotation (axis + angle) 0 Uncompressed (16 bytes)
1 3-byte axis + 2-byte angle
2 2-byte axis + 2-byte angle

3.3 Built-In Node

The Built-In Node object comprises the family of all predefined nodes in VRML (e.g. DirectionalLight, Box, etc.) Two pieces of information are required to define a builti-in node object in a VRML binary metafile: the node type, and a list of all fields present, overriding default values or not. Since a built-in node is known to the parser, the node type is represented as an index into the Built-In Node Table. The list of fields present is represented as an array of bits (the Field Mask Info) indicating the fields written in the metafile, followed by the fields themselves.

Control Code Size Info Node Index Field Mask Info Node Fields
0x2 ... ... ... ... Node fields description

Example

Built-in Node:DirectionalLight
Index into Built-In Node Table:14 (0x0E)
Field # Type Name
0 SFBool on
1 SFFloat intensity
2 SFFloat ambientIntensity
3 SFColor color
4 SFVec3f direction
ASCII 
DirectionalLight { }

Binary

Byte # Content Description
0 0x21 built-in node, 1 byte index
1 3 number of bytes remaining in the object
2 0x0E index for DirectionalLight
3 1 field mask size
4 0x00 mask: no fields present

ASCII

DirectionalLight { 
    color   1 0 0
    intensity   0.6
}

Binary

Byte # Content Description
0 0x21 built-in node, 1 byte index
1 19 number of bytes remaining in the object
2 0x0E index for DirectionalLight
3 1 field mask size
4 0x0A mask: fields 1, 3 present
5-8 0.6 intensity value, uncompressed
9-12 1.0 R color component, uncompressed
13-16 0.0 G color component, uncompressed
17-20 0.0 B color component, uncompressed
The same DirectionalLight could be represented using compressed fields. Imagining that the intensity value could be mapped to a integer in the [0, 255] range (one byte), and that the color could be mapped to a 16-bit value in 5-5-5 format, here is how that object could be represented:
Byte # Content Description
0 0x21 built-in node, 1 byte index
1 6 number of bytes remaining in the object
2 0x0E index for DirectionalLight
3 1 field mask size
4 0x0A mask: fields 1, 3 present
5 153 intensity value, compressed
6-7 0x7C00 color, compressed

ASCII

DirectionalLight { 
    on   TRUE
    direction   0.433 0.75 0.5
    ambientIntensity   0.6
}

Binary

Byte # Content Description
0 0x21 built-in node, 1 byte index
1 20 number of bytes remaining in the object
2 0x0E index for DirectionalLight
3 1 field mask size
4 0x15 mask: fields 0, 2, 4 present
5 0x01 TRUE value
6-9 0.6 intensity value, uncompressed
10-13 0.0 x vector component, uncompressed
14-17 0.0 y vector component, uncompressed
18-21 0.0 z vector component, uncompressed
The same DirectionalLight could be represented using compressed fields. Imagining that the intensity value could be mapped to a integer in the [0, 255] range (one byte), and that the direction vector could be mapped to a 16-bit value, here is how that object could be represented:
Byte # Content Description
0 0x21 built-in node, 1 byte index
1 7 number of bytes remaining in the object
2 0x0E index for DirectionalLight
3 1 field mask size
4 0x15 mask: fields 0, 2, 4 present
5 0x01 TRUE value
6 153 intensity value, compressed
7-8 0xXXXX vector, compressed

3.4 Define DEF, PROTO, Field or SFString Name

These types of nodes are some of the most commonly used nodes in a VRML binary metafile. When parsing a VRML binary metafile, a "string" name may be defined that may later be used as a PROTO name, a PROTOEXTERN name, a DEF name, a field name (in a PROTO or PROTOEXTERN node), or as a SFString name. Once a string has been defined as particular value, it retains that value unless it is redefined later in the metafile. There is no stacking mechanism for these definitions. A "Define Name" object contains the index assigned to the name, the number of bytes required to specify the string in UTF-8 form, followed by the string itself. The index and the number of bytes in the string are variable-length fields, each represented by one to eight bytes. Here is what a define "name" object looks like:

Control Code Size Info Index Info Length info String
0x8 ... ... ... ... ... ... UTF-8 data

3.5 Field Reference

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.6 SFString Reference

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.7 USE Node

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.8 ROUTE Declaration

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.9 PROTO Node Definition

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.10 EXTERNPROTO Node Definition

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.11 DEF Built-In Node

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.12 DEF Custom Node

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description

3.13 Table Of Contents (TOC) Object

TBD TBD TBD TBD
Control CodeSize InfoObject Contents
0xX ... ... Contents description